Processing system, server system, printing device, non-transitory computer-readable storage medium storing program, and processing method

ABSTRACT

A processing system includes an acquisition portion that acquires information based on a captured image of a user, and a processor that performs a login process and a logout process on an electronic device. When it is determined that a gesture of a user for instructing to log out from the electronic device is performed based on the captured image of the user, the processor performs the logout process.

The present application is based on, and claims priority from JPApplication Serial Number 2021-046970, filed Mar. 22, 2021, thedisclosure of which is hereby incorporated by reference herein in itsentirety.

BACKGROUND 1. Technical Field

The present disclosure relates to a processing system, a server system,a printing device, a non-transitory computer-readable storage mediumstoring a program, and a processing method.

2. Related Art

In the related art, a processing system is known that permits a user tolog in to an electronic device when the user is authenticated by faceauthentication. JP-A-2019-28719 discloses a method of automaticallylogging out from an electronic device after a certain period of timeelapses from the time when a user cannot be authenticated.

When logout is uniformly performed based on the fact that the electronicdevice cannot be authenticated for a certain period of time, theelectronic device logs out even when the user temporarily leaves whileusing the electronic device, which goes against the intention of theuser and makes convenience lower. In the method described inJP-A-2019-28719, such a situation is not taken into consideration.

SUMMARY

According to a first aspect of the present disclosure, there is provideda processing system including an acquisition portion that acquiresinformation based on a captured image of a user, and a processingportion that performs a login process and a logout process on anelectronic device, in which when it is determined that a gesture forinstructing to log out from the electronic device is performed by theuser based on the captured image of the user, the processing portionperforms the logout process.

According to a second aspect of the present disclosure, there isprovided a server system including the processing system according tothe first aspect, and a communicator that communicates with theelectronic device.

According to a third aspect of the present disclosure, there is provideda printing device including the processing system according to the firstaspect and a printing portion.

According to a fourth aspect of the present disclosure, there isprovided a non-transitory computer-readable storage medium storing aprogram, the program causing a computer to function as an acquisitionportion that acquires information based on a captured image of a userand a processing portion that performs a login process and a logoutprocess on an electronic device, in which when it is determined that agesture of the user for instructing to log out from the electronicdevice is performed by information based on the captured image of theuser, the processing portion performs the logout process.

According to a fifth aspect of the present disclosure, there is provideda processing method including performing a login process on anelectronic device, and performing a logout process on the electronicdevice when information based on a captured image of a user is acquiredand it is determined that a gesture of the user for instructing to logout from the electronic device is performed by information based on thecaptured image of the user.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram showing a configuration example of aprocessing system.

FIG. 2 is a diagram illustrating a processing procedure of theprocessing system.

FIG. 3 is a flowchart illustrating a login process.

FIG. 4 is a flowchart illustrating a logout process.

FIG. 5 is a block diagram showing a more detailed configuration exampleof the processing system.

FIG. 6 is a diagram illustrating a processing procedure of theprocessing system in more detail.

FIG. 7 is a flowchart illustrating a login process according to anotherexample.

FIG. 8 is a flowchart illustrating a first login process.

FIG. 9 is a flowchart illustrating a second login process.

FIG. 10 is a flowchart illustrating a logout process according toanother example.

FIG. 11 is a flowchart illustrating a first logout process.

FIG. 12 is a flowchart illustrating a second logout process.

FIG. 13 is a diagram illustrating an example of a screen to which amethod of the present embodiment is applied.

FIG. 14 is a diagram illustrating an example of the screen.

FIG. 15 is a block diagram illustrating a modification example of thepresent embodiment.

DESCRIPTION OF EXEMPLARY EMBODIMENTS

Hereinafter, the present embodiment will be described. It should benoted that the present embodiment described below does not unreasonablylimit the content of the present disclosure described in the claims.Further, not all of the configurations described in the presentembodiment are essential constituent requirements of the presentdisclosure.

FIG. 1 is a block diagram illustrating a processing system 10 of thepresent embodiment. The processing system 10 according to the presentembodiment includes an acquisition portion 110 and a processing portion120. The acquisition portion 110 is a communication interface foracquiring information based on a captured image of a user. Here, it doesnot matter whether the communication is wired communication or wirelesscommunication. The information based on the captured image of the userincludes, for example, authentication result information for faceauthentication based on the captured image of the user, but may be thecaptured image itself or other information. The authentication resultinformation for the face authentication based on the captured image ofthe user is specifically information about the result of performingauthentication on the captured face image of the user through a faceauthentication process. Further, although not shown, the processingsystem 10 may include a predetermined electronic device, and thepredetermined electronic device may further include a display. Thepredetermined electronic device is, for example, a printing device, butmay be another device.

The printing device here includes a multifunction device. Themultifunction device refers to a printer including a function other thanthe printer function, and the function other than the printer functionmay be a copy function, a fax function, a scanner function, or the like,but other functions may be also possible.

The processing portion 120 controls each portion in the processingsystem 10, such as the acquisition portion 110, performs a login processand a logout process on the above-mentioned electronic device, andperforms an execution of a job. Here, the job is, for example, in thecase where the above-mentioned electronic device is the printing device,a print job transmitted to and accumulated in the printing device, butmay be a command, an instruction, or the like given to anotherelectronic device. In the following description, the job can beappropriately read as a print job. The processing portion 120 isspecifically a processor or a controller. The processing portion 120 is,for example, a main central processing unit (CPU), but may include aplurality of CPUs such as sub CPUs. In this case, the main CPU controlseach portion of the processing system 10 and controls the whole system.The sub CPU is, for example, a CPU that controls communication of acommunicator 130 to be described later in FIG. 5. When the processingsystem 10 includes the above-mentioned printing device, the processingsystem 10 may further include a CPU that controls the printing device.Further, the processing portion 120 of the present embodiment can beconstituted by the following hardware. The hardware can include at leastone of a circuit that processes a digital signal and a circuit thatprocesses an analog signal. The hardware can be constituted, forexample, by one or more circuit devices or one or more circuit elementsmounted on a circuit board. The circuit device is, for example, anintegrated circuit device. Further, the circuit element is, for example,an active element such as a transistor or a passive element such as aresistor or a capacitor.

The face authentication process is a process of determining the validityof a person to be authenticated by comparing a feature quantity of theface image registered in advance with a feature quantity of the faceimage of the person to be authenticated at the time of authentication.Various processes can be mentioned in the face authentication process,and a process may be exemplified in which a facial part is detected foreach of the captured face image of the user and the registered faceimage, facial feature points of each facial parts are detected based onthe detected facial parts, and then the similarity of the facial partsare calculated based on the detected facial feature points. When thecalculated similarity exceeds a predetermined threshold value, it isdetermined that the user of the captured face image and the person ofthe registered face image are the same person.

The detection of the facial part can be performed, for example, by usinga method of discriminating between the face and non-face with a supportvector machine for each of the captured face image of the user and theregistered face image, but may also be performed by using a method ofdiscriminating between the face and non-face using a deep neural networkand a general learning vector quantization method.

The detection of facial feature points may be performed, for example, bya method of extracting feature points of eyes, a nose, a mouth, or thelike, from the facial part, calculating a positional relationship of thefeature points and predetermined characteristics in the vicinity of thefeature points as feature quantities, and using the features as featurequantity vectors, but may be performed by other methods. Thepredetermined characteristics are, for example, shades and distributionof color.

The calculation of the similarity of the facial part is performed, forexample, by a method of calculating the chi-square distance, theEuclidean distance, or the like, between the calculated feature quantityvectors for each of the captured face image of the user and theregistered face image, but may be performed by other methods.

The face authentication process can be performed, for example, bypredetermined hardware such as an image capturing device 20 describedlater in FIG. 5 and a first server 30, but the details thereof will bedescribed later. In this case, the information based on the capturedimage acquired by the acquisition portion 110 is authentication resultinformation for the face authentication based on the captured image ofthe user. Further, the processing portion 120 may be predeterminedhardware that executes a program for the face authentication process. Inthis case, the information based on the captured image acquired by theacquisition portion 110 is data on the captured image itself transmittedfrom hardware having the image capturing function.

Next, processing procedure of the login process and the logout processof the processing system 10 of the present embodiment will be described.The processing system 10 performs a process of transmitting a logincommand to a predetermined electronic device in response to receiving apredetermined login request. An entity that transmits the predeterminedlogin request may be the predetermined electronic device, but may beanother device as long as it is a device associated with thepredetermined electronic device. Similarly, the processing system 10performs a process of transmitting a logout command to a predeterminedelectronic device in response to receiving a predetermined logoutrequest. The predetermined login request and the predetermined logoutrequest include authentication information input by a contact-type inputmeans such as a button. Further, the processing system 10 can acceptinformation based on the captured image as the login request andtransmit the login command to the predetermined electronic device, andcan perform a process such as the login process (step S10) based on faceauthentication to be described later in FIG. 3. Similarly, theprocessing system 10 can accept information based on the captured imageas the logout request and transmit the logout command to thepredetermined electronic device, and can perform a process such as thelogout process (step S30) to be described later in FIG. 4.

Further, the logout process (step S30) described later in FIG. 4 can becombined with at least one of the predetermined login request describedabove or the login process (step S10). That is, the processing system 10performs the logout process (step S30) after transmitting the logincommand to the predetermined electronic device by the predeterminedlogin request through a contact method, and thus the processingprocedure as shown in FIG. 2 can be obtained.

FIG. 2 is a diagram illustrating a processing procedure when theabove-mentioned login process (step S10) and logout process (step S30)are combined. In FIG. 2, the downward direction indicates a time-flowingdirection, but does not indicate the specific length of time. Theacquisition portion 110 receives the information based on the capturedimage corresponding to the login process (step S10) at a predeterminedtiming. Then, the processing portion 120 performs the login process(step S10). Then, the acquisition portion 110 receives the informationbased on the captured image corresponding to the logout process (stepS30) at a timing after the predetermined timing. Then, the processingportion 120 performs the logout process (step S30).

The details of the process of FIG. 2 will be described with reference toFIGS. 3 and 4. FIG. 3 is a flowchart illustrating a processing exampleof the login process (step S10). The processing portion 120 performs aprocess of determining whether or not the user is a registered user byface authentication, and when it can be determined that the user is aregistered user (YES in step S20), a process of accepting login (stepS22) is performed. For example, the first server 30 to be describedlater in FIG. 5 performs a face authentication process, the acquisitionportion 110 acquires information indicating the determination that thecaptured person is the same person as the registered user, and theprocessing portion 120 performs a process of transmitting a command tolog in to the above-mentioned printing device or the like based on theinformation. Further, for example, the processing portion 120 mayperform the face authentication process, determine that the capturedperson is the same person as the registered user, and perform a processof transmitting the command to log in to a printing device or the like.

Then, the processing portion 120 performs a process (step S24) oftransmitting screen information according to a job situation, and endsthe procedure. The job situation is, for example, the number of jobs,but is not limited thereto, and may be a job status or the like. Forexample, when the job situation is the number of jobs, the user mayappropriately decide whether or not to include the jobs with errorsduring transmission in the number of jobs. Further, a predeterminedprinting situation may be taken into consideration in the job situation.The predetermined printing situation is, for example, a situation suchas the remaining amount of ink contained in the printing device. Forexample, as will be described later in FIG. 13 and the like, theprocessing portion 120 performs a process of transmitting, to theprinting device, a command to display different types of screeninformation according to the number of print jobs related to the userbased on the face authentication.

On the other hand, when it cannot be determined that the user is aregistered user (NO in step S20), the processing portion 120 performs aprocess of rejecting the login (step S26) and ends the procedure. Theprocess of rejecting the login (step S26) is, for example, a process ofdiscarding data on the captured image, but the process is not limitedthereto, and a process of reporting an error may be added.

As described above, the processing system 10 of the present embodimentincludes the acquisition portion 110 that acquires authentication resultinformation for face authentication based on the captured image of theuser, and the processing portion 120 that performs a process for a printjob. When it is determined that the user successfully logs in by theface authentication based on the authentication result information (YESin step S20), the processing portion 120 performs a process (step S24)of displaying a screen having a different display mode on a display ofthe printing device according to the situation of the print jobcorresponding to the user.

As described above, since the processing system 10 of the presentembodiment includes the acquisition portion 110 and the processingportion 120, it is possible to process the authentication result of faceauthentication based on the captured image. Further, the processingportion 120 can display appropriate screen information on the display ofthe printing device according to the job situation. This makes itpossible to improve the convenience of the printing device. For example,it is clear that, when the user logs in to the printing device with onlyone print job accumulated, the one accumulated print job will beprinted, and thus it is convenient for the user to execute printing ofthe print job when the login is accepted. On the other hand, when theuser logs in to the printing device with a plurality of print jobsaccumulated, in some cases, the user may not intend to print all theaccumulated print jobs. Alternatively, there is a possibility that theuser intends to delete the print job transmitted by mistake by theoperation of the printing device. In this case, it is not appropriate toprint all print jobs uniformly at the same time as accepting login.Further, even if the printing device accumulates one print job, it isnot appropriate to execute printing in a situation where the inkremaining amount of the printing device is not sufficient. In thatrespect, by applying the method of the present embodiment, it ispossible to display appropriate screen information according to the jobsituation.

Moreover, the method of the present embodiment may be implemented by aprogram. That is, the program of the present embodiment causes acomputer to function as the acquisition portion 110 that acquiresauthentication result information for face authentication based on thecaptured image of the user, and the processing portion 120 that performsa process for a print job. Further, when it is determined that the usersuccessfully logs in by the face authentication based on theauthentication result information, the processing portion 120 performs aprocess of displaying a screen having a different display mode on adisplay of the printing device according to the situation of the printjob corresponding to the user. In this way, the same effect as describedabove can be obtained.

FIG. 4 is a flowchart illustrating a processing example of the logoutprocess (step S30). The processing portion 120 performs a process (stepS40) of determining whether or not information based on the capturedimage is a gesture indicating a logout instruction. Specifically, theprocessing system 10 stores, for example, information about a gestureindicating the logout instruction in a memory (not shown) in advance.Then, the user makes a predetermined gesture by using the hardware usedfor face authentication. Then, the processing portion 120 determineswhether or not a similarity between the captured image based on thegesture of the user and the information about the stored gesture exceedsa predetermined threshold value, by the above-mentioned process. When itcan be determined that the gesture indicates the logout instruction (YESin step S40), the processing portion 120 performs a process of acceptinglogout (step S42) and ends the procedure. Specifically, when theabove-mentioned similarity exceeds the predetermined threshold value,the processing portion 120 performs a process of transmitting a commandto log out to the above-mentioned printing device. On the other hand,when it cannot be determined that the information based on the capturedimage is the gesture indicating the logout instruction (NO in step S40),the processing portion 120 performs a process of rejecting the logout(step S46) and ends the procedure. The process of rejecting the logout(step S46) is, for example, a process of discarding the data on thecaptured image data about the gesture, but a process of reporting anerror may be added.

Here, the gesture is, for example, an action of shaking the hand fromside to side and front and rear, or bending and stretching a finger;however, the gesture is not limited thereto, and may be a predeterminedfacial expression, a movement of the neck or shoulder, or apredetermined pose. The predetermined facial expression is, for example,making a surprised expression by opening the eyes or mouth, closing oneeye, or blinking a predetermined number of times. The predetermined poseis, for example, making the same shape as a character such as analphabet with a finger for a certain period of time.

As described above, the processing system 10 of the present embodimentincludes the acquisition portion 110 that acquires information based onthe captured image of the user, and the processing portion 120 thatperforms the login process and the logout process (step S30) for theelectronic device. When it is determined that the gesture of the userfor instructing to log out from the electronic device is performed basedon the captured image of the user, the processing portion 120 performsthe logout process (step S30). As described above, since the processingsystem 10 of the present embodiment includes the acquisition portion 110and the processing portion 120, the processing system 10 may perform theprocess of transmitting the login or logout instruction to apredetermined electronic device based on information based on thecaptured image. Further, since the processing portion 120 performs thelogout process when it is determined that the gesture is performed, theuser can execute logout based on his/her own intention withoutcontacting the electronic device at the timing when he/she wants to endthe use of the electronic device. In the related art, since theelectronic device logs out after a certain period of time has elapsedsince the user went out of an image-capturing range, when the user goesout of the image-capturing range while using the predeterminedelectronic device, unintended logout may occur, which may result in aproblem with convenience. In that respect, by applying the method of thepresent embodiment, the user can intentionally log out at the timingwhen he/she wants to end the use of the predetermined electronic device.There has never been such a processing system.

Moreover, the method of the present embodiment may be implemented by aprogram. That is, the program of the present embodiment causes acomputer to function as the acquisition portion 110 that acquiresinformation based on the captured image of the user, and the processingportion 120 that performs the login process and the logout process onthe electronic device. Further, when it is determined that the gestureof the user for instructing to log out from the electronic device isperformed based on the information based on the captured image of theuser, the processing portion 120 performs the logout process. In thisway, the same effect as described above can be obtained.

Moreover, the method of the present embodiment may be implemented by aprocessing method. That is, the processing method of the presentembodiment includes performing a login process on an electronic device,and performing a logout process on the electronic device wheninformation based on a captured image of a user is acquired and it isdetermined that a gesture of the user for instructing to log out fromthe electronic device is performed by information based on the capturedimage of the user. In this way, the same effect as described above canbe obtained.

Further, as shown in FIG. 2, the processing portion 120 may perform thelogin process (step S10) by information based on the captured imagecorresponding to the login process (step S10), and then performs thelogout process (step S30) by information based on the captured imagecorresponding to the logout process (step S30). That is, when it isdetermined that the gesture for instructing logout is performed afterthe login of the user is performed by the face authentication processbased on the captured image, the processing portion 120 performs thelogout process. By doing so, the user who logs in based on thenon-contact face authentication can log in to and out from thepredetermined electronic device without contact by logging out based onthe non-contact gesture. In particular, in a situation where there are aplurality of types of authentication means, a user logged in by faceauthentication may have an intention to log in without contact. In thisrespect, by applying the method of the present embodiment, it ispossible to log out without contact, which improves convenience.

The processing system 10 of the present embodiment is not limited to theabove-described configuration, and various modifications such as addingother components can be performed. FIG. 5 is a block diagramillustrating a more detailed configuration example of the processingsystem 10 of the present embodiment. The processing system 10 of FIG. 5includes an image capturing device 20, a first server 30, a secondserver 100, an electronic device 200, and a terminal device 300. Thesecond server 100 includes the acquisition portion 110 and theprocessing portion 120 described above, and further includes acommunicator 130. The electronic device 200 includes a display 240. Thecommunicator 130 is an interface for communication connection with theelectronic device 200 and the terminal device 300, but may be the sameinterface as the acquisition portion 110 described above. The electronicdevice 200 is, for example, a printing device, but may be anotherdevice. When the electronic device 200 is a printing device, it furtherincludes a printing portion (not shown).

The functions of the acquisition portion 110 and the processing portion120 in FIG. 5 correspond to some of the functions of the acquisitionportion 110 and the processing portion 120 of FIG. 1, respectively.Therefore, the second server 100 can be considered as a server system inwhich the processing system 10 of FIG. 1 is further provided with thecommunicator 130. That is, the server system of the present embodimentincludes the above-mentioned processing system 10 and the communicator130 that communicates with the electronic device 200. By doing so, it ispossible to construct a server system that acquires information based onthe captured image and performs the login process and the logout processon the electronic device 200.

The image capturing device 20 is located at a predetermined locationnear the electronic device 200, for example, captures an image of a userwho wants to operate the electronic device 200, and transmitsinformation on a captured image to the first server 30 to be describedlater. The image capturing device 20 and the electronic device 200 donot need to be directly connected to each other by communication. Thisis because, since the information on the captured image includes theidentification information for the image capturing device 20 itself, theuser can determine which the image capturing device 20 corresponds towhich electronic device 200 by associating the identificationinformation for the image capturing device 20 with the identificationinformation for the electronic device 200. The image capturing device 20can be built by, for example, a near-infrared camera, but may be anothercamera. Further, the image capturing device 20 may be formed of aplurality of cameras.

In the case of FIG. 5, the first server 30 performs face authenticationbased on the above-mentioned information on the captured image, andtransmits the information based on the captured image to the secondserver 100. The information based on the captured image isauthentication result information for the face authentication. The firstserver 30 includes a memory (not shown). For example, the memory storesimage information about the face of a registered user used for a faceauthentication program or face authentication, but may also store imageinformation about the above-mentioned gesture. The first server 30 canperform the face authentication process by including hardware such as aprocessor or a controller in addition to the above-mentioned memory.

The second server 100 receives information based on the captured imagefrom the first server 30, and determines whether to accept login orlogout for the electronic device 200 corresponding to the imagecapturing device 20. Further, the second server 100 receivespredetermined job data from the terminal device 300 and stores it in amemory (not shown). The predetermined job data is, for example, printjob data when the electronic device 200 is a printing device. Thepredetermined job data transmitted by the terminal device 300 includesuser identification information. Therefore, when the user logs in to theelectronic device 200, the job information about the logged-in user isdisplayed on the display 240.

The display 240 can be built by, for example, a liquid crystal display,but may be built by an organic electroluminescent (EL) display, a dotmatrix light-emitting diode (LED), or the like. Further, the display 240may be integrated hardware with an operation portion (not shown) and atouch panel, or may give the user a report on the electronic device 200together with a report portion (not shown). In the present embodiment,the display 240 is a touch panel.

FIG. 6 is a diagram illustrating a processing procedure of the entireprocessing system 10 in FIG. 5. Here, it is assumed that the electronicdevice 200 is a printing device. The terminal device 300 transmits aprint job to the second server 100, and the second server 100 performs aprint job storage process. Then, the second server 100 receives either afirst login request or a second login request. The first login requestis authentication information transmitted from the first server 30 basedon information on the captured image transmitted from the imagecapturing device 20. The second login request is authenticationinformation that is not based on the information on the captured image,for example, authentication information transmitted by the userinputting and operating a predetermined operation portion, orauthentication information by using a predetermined electronic medium.The predetermined operation portion is, for example, a button (notshown), but may be a touch panel of the display 240.

After receiving the first login request or the second login request, theprocessing portion 120 performs a login process (step S100) on theprinting device. Although both the first login request and the secondlogin request are shown in FIG. 6, one of them is actually transmittedto the second server 100. The same is applied to a first logout requestand a second logout request to be described later. Then, when a printdata request (step S200) based on job information about the userdisplayed on the display 240 is received from the printing device, theprocessing portion 120 transmits print data (step S210). In this way,the printing device executes printing based on the received print data,and the user can obtain a desired printed matter. Depending on the jobsituation, the print data may be transmitted to the printing device evenif there is no print data request (step S200), and the details will bedescribed later with reference to FIG. 8.

Then, the second server 100 receives either the first logout request orthe second logout request. The first logout request is authenticationinformation transmitted from the first server 30 based on information ona gesture image transmitted from the image capturing device 20. Thesecond logout request is authentication information that is not based onthe information on the captured image, like the second login request.After receiving the first logout request or the second logout request,the processing portion 120 performs a logout process (step S300) on theprinting device.

Next, the specific content of each process of FIG. 6 will be describedwith reference to FIGS. 7, 8, 9, 10, 10, 11, and 12. Here, a descriptionwill be given including a processing example in which a screen displayafter login by face authentication is different depending on the numberof print jobs, which is an example of a print job status. That is, theprocessing portion 120 performs a process of displaying the screenshaving different display modes depending on the number of print jobs onthe display 240. By doing so, the electronic device 200 can displayappropriate screen information on the display 240 depending on thenumber of print jobs.

FIG. 7 is a flowchart illustrating the login process (step S100). Theprocessing portion 120 performs a process of determining whether thelogin request is the first login request or the second login request,and when the login request is the first login request (YES in stepS110), performs the first login process (step S120) and ends theprocedure. On the other hand, when the login request is the second loginrequest (NO in step S110), the processing portion 120 performs thesecond login process (step S220) and ends the procedure.

FIG. 8 is a flowchart illustrating the first login process (step S120).The processing portion 120 performs user authentication (step S122).When there is a problem with user authentication, the process will notproceed to a subsequent process, but here it is assumed that there is noproblem with user authentication and the process will proceed. The sameis applied to the description of FIGS. 9, 13, and 14 to be describedlater. Then, the processing portion 120 performs a process ofdetermining whether or not the number of print jobs of the useraccumulated in the printing device is zero, performs a process (stepS132) of transmitting, to the printing device, an instruction to causethe display 240 to display in a first display mode when the number ofprint jobs is zero (YES in step S130), and then performs processes ofstep S160 and subsequent steps. On the other hand, when the number ofprint jobs is not zero (NO in step S130), the processing portion 120performs a process of determining whether or not the number of printjobs is one, performs a process (step S144) of transmitting, to theprinting device, an instruction to cause the display 240 to display inthe second display mode when the number of print jobs is one (YES instep S140), performs a process (step S142) of transmitting print jobdata to the printing device, and then performs processes of step S160and subsequent steps. Step S142 is a process corresponding to step S210described above in FIG. 6, and the printing device that receives theprint job data transmitted in step S142 executes printing based on theprint job data. As described above, the print job data may betransmitted to the printing device even if the above-mentioned step S200is not performed in FIG. 6. Further, when the number of print jobs istwo or more (NO in step S140), the processing portion 120 performs aprocess (step S150) of transmitting, to the printing device, aninstruction to cause the display 240 to display in a third display mode,and then performs processes of step S160 and subsequent steps.

As described above, in the first login process (step S120), theprocessing portion 120 of the present embodiment performs processes ofdividing the number of jobs into three patterns and displaying, on thedisplay 240, screens having different display modes for each pattern.That is, when the number of print jobs for the user when the login issuccessful is zero, the processing portion 120 displays a screen of thefirst display mode. Moreover, when the number of print jobs for the userwhen the login is successful is one, the processing portion 120 displaysa screen of the second display mode. Moreover, when the number of printjobs for the user when the login is successful is two or more, theprocessing portion 120 displays a screen of the third display mode. Bydoing so, it is possible to perform an appropriate guide displayaccording to the number of print jobs.

Specifically, the processing portion 120 of the second server 100 as aserver system generates display screen data depending on the firstdisplay mode to the third display mode. Then, the processing portion 120performs a process of transmitting the display screen data to theelectronic device 200 as a printing device via the communicator 130 at apredetermined timing. The same is applied to step S232 and step S250 inFIG. 9. That is, the server system of the present embodiment includesthe above-mentioned processing system 10 and the communicator 130 thatcommunicates with the electronic device 200 that is a printing device,and the processing portion 120 gives an instruction to display a screenhaving a different display mode on the display 240 of the electronicdevice 200, which is the printing device, according to the situation ofthe print job, via the communicator 130. By doing so, it is possible toconstruct a server system that generates an appropriate guide displayand displays the screen of the guide display on the printing device. Aspecific screen example of the first display mode to the third displaymode will be described later with reference to FIG. 13.

Then, the processing portion 120 performs a process (step S160) ofinvalidating the second logout process (step S330) to be described laterin FIG. 12. That is, when the first login process (step S120) based onthe captured image is performed as the login process, the processingportion 120 invalidates the second logout process (step S330). By doingso, the user who selects to use the electronic device 200 based on thenon-contact-type login can be prevented from logging out by the contactmethod.

Specifically, a process of generating image data that does not reflect alogout button icon 540 to be described later in FIG. 13 or the like inthe above-mentioned first display mode to the third display mode isperformed. That is, the processing portion 120 performs the first loginprocess (step S120) based on the captured image and the second loginprocess (step S220) based on information other than the captured image,as the login process. Then, when the first login process (step S120) isperformed, the processing portion 120 invalidates display of aninstruction of the second logout process (step S330) corresponding tothe second login process (step S220) on the display 240 of theelectronic device 200. By doing so, the user can confirm that he/shelogs in in a non-contact manner.

Then, the processing portion 120 performs a process (step S162) ofdisplaying the guide display in a superimposed manner. Specifically, aswill be described later in FIG. 13, the processing portion 120 performsa process of generating image data in which a logout method guide icon530 is reflected in the above-mentioned first display mode to the thirddisplay mode. Since step S162 is a process performed when login isperformed by face authentication, displaying the logout method guideicon 530 described later in FIG. 13 means that face authentication issuccessful. That is, when the face authentication is successful, theprocessing portion 120 of the present embodiment performs a displayprocess of notifying that the face authentication is successful, on thedisplay 240 of the printing device. By doing so, the user can confirmthat he/she logs in by face authentication.

FIG. 9 is a flowchart illustrating the second login process (step S220).The processing portion 120 performs user authentication (step S222) inthe same manner as in step S122 described above. Then, the processingportion 120 performs a process of determining whether or not the numberof print jobs of the user accumulated in the printing device is zero,performs a process (step S232) of displaying in a first display modewhen the number of print jobs is zero (YES in step S230), and thenperforms a process of step S260. On the other hand, when the number ofprint jobs is not zero (NO in step S230), the processing portion 120performs a process (step S250) of transmitting, to the printing device,an instruction to cause the display 240 to display in a third displaymode, and then performs a process of step S260.

Then, the processing portion 120 performs a process (step S260) ofvalidating the second logout process (step S330) to be described laterin FIG. 12. Specifically, as will be described later with reference toFIG. 14, a process of transmitting, to the printing device, aninstruction to display the logout button icon 540 on the display 240 isperformed.

From FIGS. 8 and 9, when the first login process (step S120) isexecuted, the user can only log out by the first logout process (stepS320). Further, since the second login process (step S220) does not havethe process corresponding to step S162 in FIG. 8, the logout methodguide icon 530 to be described later in FIG. 13 is not displayed.Therefore, the user cannot recognize the logout method by the gesture,and naturally prompts the logout by the second logout process (stepS330). As described above, when the first login process (step S120)based on the captured image is performed as the login process, theprocessing portion 120 of the present embodiment performs the firstlogout process (step S320) based on the captured image about the gestureas the logout process. Further, when the second login process (stepS220) based on the information other than the captured image isperformed, the processing portion 120 performs the second logout process(step S330) based on the information other than the captured image asthe logout process. By doing so, the method of logging in to and outfrom the electronic device 200 can be matched to either the non-contactmethod or the contact method.

FIG. 10 is a flowchart illustrating the logout process (step S300). Theprocessing portion 120 performs a process of determining whether or notthe logout request is the first logout request based on information onthe gesture image, performs the first logout process (step S320) whenthe logout request is the first logout request (YES in step S310), andends the procedure. On the other hand, the processing portion 120performs the second logout process (step S330) when the logout requestis not the first logout request, that is, the second logout request (NOin step S310), and ends the procedure.

FIG. 11 is a flowchart illustrating the first logout process (stepS320). The processing portion 120 performs a process (step S322) ofdetermining whether or not the information on the gesture imagesatisfies a first logout condition. The information on the gesture imagehere includes information based on the gesture image which is a capturedimage. The first logout condition may be, for example, that thesimilarity between the gesture image based on the gesture performed bythe user and the registered gesture image exceeds a predeterminedthreshold value. Further, for example, when the information based on thegesture image is a character or a number, the first logout condition maybe that the character or the number based on the captured gesture imagematches the registered character or the number. When it is determinedthat the information on the gesture image satisfies the first logoutcondition (YES in step S322), the processing portion 120 performs aprocess of accepting logout (step S324) and ends the procedure. Forexample, the processing portion 120 performs a process of generatingscreen display data before the above-mentioned login process (step S100)is performed, and a process of transmitting the screen display data tothe printing device.

On the other hand, when it is determined that the information on thegesture image does not satisfy the first logout condition (NO in stepS322), the processing portion 120 performs a process of rejecting thelogout (step S326) and ends the procedure. Specifically, the processingportion 120 reports that the similarity between the gesture image basedon the gesture performed by the user and the registered gesture imagedoes not exceed the predetermined threshold value, but the reporting isnot limited thereto. For example, the processing portion 120 may reportthat the characters or numbers based on the gesture performed by theuser do not match the registered characters or numbers.

A process of re-validating the above-mentioned second logout process(step S330) may be added. This is because the rejecting of the logoutmay be caused by a malfunction or the like of the processing system 10.

FIG. 12 is a flowchart illustrating the second logout process (stepS330). The processing portion 120 performs a process of determiningwhether or not information by an electronic medium or an operation ofthe user satisfies a second logout condition. That is, the second logoutprocess (step S330) is a logout by an electronic medium or a logout byan operation on the display 240 by the user. By doing so, it is possibleto log out by a method using information other than the informationbased on the captured image. The second logout condition is that, forexample, when the information based on an ID card used for logoutmatches the information registered as the logged-in user. When it isdetermined that the information by the electronic medium or theoperation of the user satisfies the second logout condition (YES in stepS332), the processing portion 120 performs a process of accepting thelogout (step S334). Specifically, the same is applied to theabove-mentioned step S324.

On the other hand, when it is determined that the information by theelectronic medium or the operation by the user does not satisfy thesecond logout condition (NO in step S332), the processing portion 120performs a process of rejecting the logout (step S336) and ends theprocedure. For example, when logout is performed with an ID card ownedby a user different from the logged-in user, a process of transmitting acommand to display the fact on the display 240 to the printing device isperformed. It is not normally assumed that the logout is rejected byselecting the logout button icon 540 to be described later.

FIG. 13 is an example of a screen to which the method of the presentembodiment is applied. When the electronic device 200 is activated bypowering on, the display 240 displays a user authentication icon 500.Then, when the user selects login by face authentication, that is, whenthe first login request is made, the processing portion 120 performs thefirst login process (step S100) described above in FIG. 8. Therefore,depending on the number of jobs, a display image according to any one ofthe above-mentioned first display mode, second display mode, and thirddisplay mode is displayed on the display 240.

A user display icon 501 is displayed on the display 240 in any of thefirst display mode to the third display mode. In this way, the user canconfirm that the user authentication (step S122) is performed. Further,as shown in A1, A2, and A3, the logout button icon 540 to be describedlater in FIG. 14 is not displayed on the display 240 in any of the firstdisplay mode to the third display mode. This is because the process ofstep S160 described in FIG. 8 is executed in any of the first displaymode to the third display mode. In addition to completely erasing thelogout button icon 540, the processing portion 120 may make the shape ofthe logout button icon 540 different from that shown in FIG. 14, such asmaking the logout button icon 540 semi-transparent, and may not acceptthe touch panel operation even if the logout button icon 540 isdisplayed.

When the number of print jobs is zero (YES in step S130), the display240 displays a print button icon 502, a FAX button icon 504, a copybutton icon 506, and a scanner button icon 508 in a list, as the firstdisplay mode. That is, the first display mode is a mode for indicating alist of functions of the printing device. In this way, when the numberof print jobs is zero for the user who successfully logs in, theprocessing portion 120 performs a process of displaying a function listscreen of the printing device. By doing so, the user can optionallyselect the content to be executed for the printing device.

When the number of print jobs is one (YES in step S140), the display 240displays an in-printing icon 510 as the second display mode. That is,the second display mode is a mode for indicating the execution ofprinting. As described above, when the number of print jobs is one forthe user who successfully logs in, the processing portion 120 executesprinting on the printing device and performs a process of displaying aprint execution screen. By doing so, the user can cause the printingdevice to execute the accumulated print job in response to the login.

When the number of print jobs is two or more (NO in step S140), thedisplay 240 displays print waiting icons 520 and 522 as the thirddisplay mode. That is, the third display mode is a mode for indicating alist of print jobs for which printing has not yet been executed. Asdescribed above, when the number of print jobs is two or more for theuser who successfully logs in, the processing portion 120 performs aprocess of displaying a list screen of a plurality of print jobs. Bydoing so, the user can appropriately respond to two or more print jobs.

The logout method guide icon 530 is displayed on the display 240 in anyof the first display mode to the third display mode. This is because theprocess of step S162 described in FIG. 8 is executed in any of the firstdisplay mode to the third display mode.

The logout method guide icon 530 is a display that guides the user tolog out by gesture when login is performed by face authentication, thatis, by the first login process (step S120). That is, the processingportion 120 performs the first login process (step S120) based on thecaptured image and the second login process (step S220) based oninformation other than the captured image, as the login process. Then,when the first login process (step S120) is performed, the processingportion 120 performs the guide display for guiding the user to performthe first logout process (step S320) by the gesture corresponding to thefirst login process (step S120) on the display 240 of the electronicdevice 200. By doing this, the user can recognize that he/she has to logout by gesture.

FIG. 14 is an example of a screen when the processing portion 120performs the second login process (step S220). In FIG. 14, the secondlogin process is different from the first login process (step S120) inthat, when the number of print jobs is one, the display is performed inthe third display mode (NO in step S230), and the print waiting icon 520is displayed on the display 240. Further, since the second logoutprocess is validated (step S260), the logout button icon 540 isdisplayed on the display 240 regardless of the number of print jobs.Further, since the process of step S162 of FIG. 8 is not performed, thelogout method guide icon 530 is not displayed on the display 240regardless of the number of print jobs, as shown in B1, B2, and B3,which is different from the first login process. The user can log outfrom the electronic device 200 by selecting the logout button icon 540by touch panel operation. That is, the logout button icon 540 is acontact-type logout means. Although not shown, the user may be able tolog out by using an ID card or the like in addition to the logout buttonicon 540.

The method of the present embodiment is not limited to the above, andvarious modifications can be implemented. FIG. 15 is a block diagramillustrating a modification example of the present embodiment. Theprocessing system 10 of FIG. 15 differs from FIG. 5 in that anelectronic device 400 also serves as the second server 100 of FIG. 5.That is, the electronic device 400 includes an acquisition portion 410,a processing portion 420, a communicator 430, and a display 440. Theacquisition portion 410 corresponds to the acquisition portion 110 ofFIG. 1, the processing portion 420 corresponds to the processing portion120 of FIG. 1, the communicator 430 corresponds to the communicator 130of FIG. 5, and the display 440 corresponds to the display 240 in FIG. 5.Further, the electronic device 400 may be, for example, a printingdevice, and in this case, further includes a printing portion (notshown). Therefore, the electronic device 400 can be considered as aprinting device in which the processing system 10 of FIG. 1 is furtherprovided with a printing portion. By doing so, it is possible toconstruct a printing device that acquires information based on thecaptured image and performs its own login process and the logoutprocess. Further, it is possible to provide a printing device thatperforms the login process based on face authentication and performs anappropriate guide display.

As described above, the processing system of the present embodimentincludes the acquisition portion that acquires information based on thecaptured image of the user, and the processing portion that performs thelogin process and the logout process on the electronic device. Further,when it is determined that a gesture of a user for instructing to logout from the electronic device is performed based on the captured imageof the user, the processing portion performs the logout process.

By doing so, the user can perform logout based on his/her own intentionwithout contacting the electronic device at the timing when he/she wantsto end the use of the electronic device. This makes it possible toimprove the convenience of the electronic device.

Further, when it is determined that the gesture for instructing logoutis performed after the login of the user is performed by the faceauthentication process based on the captured image, the processingportion may perform the logout process.

By doing so, the user who logs in based on the non-contact faceauthentication can log in to and out from the electronic device withoutcontact by logging out based on the non-contact gesture.

Further, the processing portion may perform the first logout processbased on the captured image about the gesture as the logout process whenthe first login process based on the captured image is performed as thelogin process, and may perform the second logout process based oninformation other than the captured image as the logout process when thesecond login process based on the information other than the capturedimage is performed.

By doing so, the method of logging in to and out from the electronicdevice can be matched to either the non-contact method or the contactmethod.

Further, the processing portion may invalidate the second logout processwhen the first login process based on the captured image is performed asthe login process.

By doing so, the user who selects to use the electronic device based onthe non-contact-type login can be prevented from logging out by thecontact method.

Further, the processing portion may perform the first login processbased on the captured image and the second login process based on theinformation other than the captured image as the login process, and mayinvalidate the display on the display of the electronic device thatinstructs to perform the second logout process corresponding to thesecond login process when the first login process is performed.

By doing so, the user can confirm that he/she logs in in a non-contactmanner.

Further, the second logout process may be a logout by an electronicmedium or a logout by an operation on the display by the user.

By doing so, it is possible to log out by a method using informationother than the information based on the captured image.

Further, the processing portion may perform the first login processbased on the captured image and the second login process based on theinformation other than the captured image as the login process, andperform a guide display for guiding the user to perform the first logoutprocess by the gesture corresponding to the first login process on thedisplay of the electronic device when the first login process isperformed.

By doing this, the user can recognize that he/she has to log out bygesture.

Further, the server system of the present embodiment includes theabove-mentioned processing system and the communicator that communicateswith the electronic device.

By doing so, it is possible to construct a server system that acquiresinformation based on the captured image and performs the login processand the logout process of the electronic device.

Further, the printing device of the present embodiment includes theabove-mentioned processing system and printing portion.

By doing so, it is possible to construct a printing device that acquiresinformation based on the captured image and performs the login processand the logout process on the electronic device.

Further, the program of the present embodiment relates to a program thatcauses a computer to function as the acquisition portion that acquiresinformation based on the captured image of the user, and the processingportion that performs the login process and the logout process on theelectronic device. Further, when it is determined that the gesture ofthe user for instructing to log out from the electronic device isperformed based on the information based on the captured image of theuser, the processing portion performs the logout process.

Further, the processing method of the present embodiment includesperforming a login process on an electronic device, and performing alogout process on the electronic device when information based on acaptured image of a user is acquired and it is determined that a gestureof the user for instructing to log out from the electronic device isperformed by information based on the captured image of the user.

Although the present embodiment has been described in detail asdescribed above, it will be easily understood by those skilled in theart that various modifications could be made without substantiallydeparting from the novel matters and effects of the present disclosure.Therefore, all such modification examples fall within the scope of thepresent disclosure. For example, in a specification or drawing, a termdescribed at least once with a different term having a broader meaningor a synonym may be replaced with the different term in any part of thespecification or the drawing. All combinations of the present embodimentand modification examples also fall within the scope of the presentdisclosure. Further, the configuration/operation of the processingsystem, the server system, the printing device, the program, theprocessing method, and the like is not limited to those described in thepresent embodiment, and various modifications could be made.

What is claimed is:
 1. A processing system comprising: an acquisitionportion that acquires information based on a captured image of a user;and a processor that performs a login process and a logout process on anelectronic device, wherein when it is determined that a gesture forinstructing to log out from the electronic device is performed by theuser based on the captured image of the user, the processor performs thelogout process.
 2. The processing system according to claim 1, whereinwhen it is determined that the gesture for instructing to log out isperformed after the user logs in by a face authentication process basedon the captured image, the processor performs the logout process.
 3. Theprocessing system according to claim 1, wherein the processor performs afirst logout process based on the captured image related to the gestureas the logout process when a first login process based on the capturedimage is performed as the login process, and performs a second logoutprocess based on information other than the captured image as the logoutprocess when a second login process based on the information other thanthe captured image is performed.
 4. The processing system according toclaim 3, wherein when the first login process based on the capturedimage is performed as the login process, the processor invalidates thesecond logout process.
 5. The processing system according to claim 1,wherein the processor performs a first login process based on thecaptured image and a second login process based on information otherthan the captured image as the login process, and invalidates display ofan instruction of a second logout process corresponding to the secondlogin process on a display of the electronic device when the first loginprocess is performed.
 6. The processing system according to claim 3,wherein the second logout process is logout by an electronic medium orlogout by an operation on a display by the user.
 7. The processingsystem according to claim 1, wherein the processor performs a firstlogin process based on the captured image and a second login processbased on information other than the captured image as the login processand performs a guide display for guiding the user to perform a firstlogout process by the gesture corresponding to the first login processon a display of the electronic device when the first login process isperformed.
 8. A server system comprising: the processing systemaccording to claim 1; and a communicator that communicates with theelectronic device.
 9. A printing device comprising: the processingsystem according to claim 1; and a printing portion.
 10. Anon-transitory computer-readable storage medium storing a program, theprogram causing a computer to function as: an acquisition portion thatacquires information based on a captured image of a user; and aprocessor that performs a login process and a logout process on anelectronic device, wherein when it is determined that a gesture of theuser for instructing to log out from the electronic device is performedby information based on the captured image of the user, the processorperforms the logout process.
 11. A processing method comprising:performing a login process on an electronic device; and performing alogout process on the electronic device when information based on acaptured image of a user is acquired and it is determined that a gestureof the user for instructing to log out from the electronic device isperformed by information based on the captured image of the user.